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been  defined  from  this  theoretical  framework  and  lead  to  parallelizable  implementations  based  on  FFT  and  dynamic  programming. 
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1  Objectives 

Spatio-temporal  signal  captured  by  an  array  of  sensors  like  digital  image  sequences  can  provide  useful  inputs 
to  a  wide  variety  of  automated  systems,  such  as  autonomous  vehicle  navigators,  and  surveillance  systems 
to  mention  a  few.  The  design  of  signal  processing  techniques  that  allow  motion  information  to  be  reliably 
extracted  remains  a  challenging  problem,  particularly  when  noise,  jitter,  and  temporary  occlusions  are 
present.  In  this  project,  the  PI  has  addressed  the  problem  of  motion  analysis  spaces  from  a  new  theoretical 
perspective  based  on  Lie  group  representations  to  develop  the  harmonic  analysis  associated  with  Lie  groups  of 
spatio-temporal  transformations.  The  entire  construction  of  harmonic  analysis  is  based  on  two  assumptions 
related  to  the  actual  physics  of  motion: 

1.  locally,  any  observable  motion  transformation  or  kinematic  is  described  by  a  Lie  algebra  or  a  Lie 
groups 

2.  globally,  any  trajectory  is  defined  from  a  variational  principle  of  optimality. 

The  harmonic  analysis  of  motion  transformations  fundamentally  differs  from  the  other  current  model¬ 
ings  presented  in  the  literature  which  mainly  rely  on  techniques  based  on  stochastic  processes,  statistics 
and  operations  research.  As  a  major  drawback,  these  existing  techniques  are  totally  blind  to  the  underlying 
mathematical  and  kinematical  structures  of  the  spatio-temporal  transformations.  The  research  that  has  be 
done  before  1995  to  analyze  motion  with  a  group-theoretic  point  of  view  was  only  focusing  on  biological 
perception.  In  fact,  biological  sensors  behave  differently  from  electronic  sensors. 

The  spatio-temporal  transformations  of  concern  deal  with  motions  talcing  place  in  the  outer  space  of 
observation.  The  tools  to  be  developed  analyze  these  spatio-temporal  signals  after  their  projections  in  the 
cone  of  visibility  of  sensor  arrays  (like  video  or  infra-red  cameras).  In  this  work,  the  motion  trajectories  to 
analyze  occur  on  smooth  manifolds  and  the  sensor  arrays  may  be  deployed  on  some  other  smooth  manifolds. 
Three  categories  of  motion  are  considered  and  referred  to  as  translational,  rotational  or  deformational.  In 
each  category,  motion  parameters  are  defined  from  all  the  temporal  derivatives  i.e.  position,  velocity  and 
accelerations.  Motion  analysis  means  first  of  all  selective  detection,  estimation  and  tracking.  But  it  turns 
out  that  the  continuous  wavelets  developed  in  this  program  present  other  efficient  properties  that  able  to 
cope  with  additional  applications,  namely  spatio-temporal  interpolation,  selective  motion-compensated  fil¬ 
tering,  signal  decomposition,  and  selective  reconstruction.  An  innovative  study  of  selective  motion  tracking 
is  based  on  the  adjunction  of  variational  principles  of  optimality  to  the  group  representations  in  order  to 
define  the  appropriate  equations  of  wavelet  motion,  the  tracking  equations,  selective  constants  of  motion 
to  be  tracked,  and  all  the  symmetries  to  be  imposed  on  the  analyzing  wavelets.  The  study  of  the  har¬ 
monic  analysis  associated  with  the  new  motion  groups  is  expected  to  derive  new  tools  to  be  introduced 
in  tracking  algorithms.  This  important  issue  leads  to  the  definition  of  special  functions,  propagators  and 
motion-compensated  filters.  Special  functions  provide  in  the  Fourier  domain  specific  spectral  signatures 
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based  on  specific  motion  to  detect  or  estimate.  Propagators  generate  any  motion-tuned  wavelet  from  their 
still  cognate.  Motion-compensated  filters  generate  a  still  version  from  a  moving  pattern  i.e.  unwarp  the  mo¬ 
tion  transformation.  Propagators  and  motion-compensated  filters  may  as  well  process  any  moving  pattern 
in  order  to  synthesize  a  new  version  moving  with  different  motion  characteristics. 

Nice  properties  developed  from  the  modern  abstract  algebra  apply  to  this  theoretical  framework  to 
supply  fast  and  highly  parallelizable  computational  algorithms  based  on  Fast  Fourier  Transforms  for  the 
continuous  wavelet  transforms  and  the  inverse  problem  of  reconstruction,  on  gradient  algorithms  for  the 
minimum-mean-square-error  estimation,  and  on  dynamic  programming  techniques  for  tracking. 

An  example  of  application  for  this  theoretical  framework  may  be  summarized  as  follows.  Let  us  consider 
a  passive  system  hosted  in  a  mobile  vehicle  tracking  selected  moving  targets  with  one  single  camera.  First, 
the  trajectory  is  decomposed  into  small  charts  spanning  over  a  few  seconds  covering  several  hundreds  of 
frames  (typically  100  to  1000).  Second,  we  assume  that  the  surface  on  which  motion  transformations  occur 
is  known.  In  this  case,  the  environment  is  acquired  and  described  by  a  set  of  external  surrounding  active 
sensors  (made  of  ground  radars  or  satellites)  that  transmit  the  information  to  the  algorithm  that  passively 
performs  estimation  and  tracking.  The  information  supplied  about  the  target  contains  surface  (geometry  as 
curvature)  and  motion  (initial  or  updated  moving  conditions).  At  the  beginning  of  each  chart,  the  passive 
tracking  system  receives  from  its  base  the  update  of  the  kinematics.  From  the  initial  kinematical  conditions 
on  the  chart,  the  algorithm  is  capable  to  adjust  its  tracking  and  decides  which  wavelet  family  to  use  to  keep 
on  performing  motion  estimation  and  tracking  until  the  next  update. 

2  Past  accomplishments  on  the  topic  (not  covered  by  AFOSR) 

2.1  Period  11-1-93  to  2-1-96 

During  that  period,  the  PI  combined  discrete  spatio-temporal  wavelet  filtering  with  motion  analysis  algo¬ 
rithms  based  on  operations  research  methods  and  statistics  for  image  sequence  processing.  The  PI  developed 
the  technique  of  motion-compensated  filtering  which  means  filtering  along  the  assumed  motion  trajectories. 
The  main  problem  in  this  approach  was  the  huge  computational  load  of  the  algorithms  to  estimate  the 
motion  parameters  and  compute  the  trajectories.  The  new  idea  was  to  design  wavelet  transforms  as  families 
of  convolutional  filters  tuned  to  capture  and  extract  objects  in  motion  out  of  a  scene.  The  wavelet  should 
be  continuous  to  provide  a  continuous  range  of  motion  parameter  estimation.  The  first  parameters  to  be 
considered  were  the  scale,  the  spatio-temporal  location,  the  orientation  and  the  velocity.  This  refers  to  the 
so-called  Galilei  group  known  by  physicists. 

2.2  Period  5-1-  96  to  6-1-97 

The  purpose  of  this  research  work  was  to  develop  an  application  of  the  Galilean  wavelet  transforms  to 
perform  automatic  target  recognition.  The  particular  problem  addressed  in  this  research  was  the  tracking 
of  a  warhead  for  an  incoming  ballistic  missile. 

During  that  period,  the  PI  has  evidenced  the  MMSE  property  of  the  continuous  wavelet  transforms  and 
their  matching  with  Kalman  filters.  Dynamic  programming  algorithms  have  been  used  to  implement  the 
tracking  algorithms.  These  algorithms  were  based  on  FFT’s  and  optimal  control  with  highly  parallelizable 
schemes. 

3  Final  Status  of  Effort 

During  the  initial  period  of  this  project,  the  PI  started  to  investigate  the  existence  of  new  continuous  wavelet 
transforms  for  accelerated  translational  motion,  for  rotational  motion  (angular  velocity  and  accelerations), 
and  for  deformational  motion  (temporal  derivatives  of  the  change  of  scale).  For  these  transformations,  the 
PI  demonstrated  the  existence  of  Lie  algebras,  Lie  groups,  Lie  group  representations,  and  eventually  the 
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existence  of  continuous  wavelets  for  sets  of  motion  parameters  whose  dimension  is  not  larger  than  the  di¬ 
mension  of  the  phase  space  (i.e.  the  Fourier  space  of  translation  and  central  parameters). 

From  these  new  findings,  the  PI  kept  on  examining  more  general  cases  dealing  with  the  analysis  of  mo¬ 
tion  on  smooth  manifolds.  At  this  level  of  generalization,  the  PI  evidenced  the  importance  of  extending  and 
deforming  Lie  algebras:  this  enables  to  visit  all  the  physically  possible  and  admissible  kinematics  according 
to  time  and  parity  reversal.  These  kinematics  are  also  those  embedded  in  digital  signals.  Starting  with  ad¬ 
missible  Lie  algebras,  the  construction  of  the  group  composition  is  made  possible  using  the  Baker-Hausdorff 
formulas.  The  group  representations  are  thereafter  computed  by  the  usual  method  developed  by  Mackey, 
Auslander,  Pukansky  and  Kirillov.  These  group  representations  for  motion  on  smooth  manifolds  remain 
subject  to  generalized  square-integrability  conditions.  They  give  rise  to  generalized  Fourier  transforms  to 
be  considered  as  integral  equations  whose  properties  remain  to  be  studied.  Moreover,  as  curvature  may 
vary  from  one  area  to  the  other,  we  may  consider  the  Cartan  geometry  defining  bundles  of  Lie  algebras, 
kinematics,  Lie  groups,  continuous  wavelets  and  integral  transforms. 

The  problem  of  the  cone  of  visibility  of  an  array  of  sensor  is  solved.  It  consists  to  relate  Fourier  trans¬ 
forms,  group  representations,  Lie  groups,  and  Lie  algebras  in  the  outer  space  i3  xl  with  their  cognates 
in  the  captured  signals  in  R2  x  R.  The  assumptions  are  as  follows.  First,  the  motion  in  the  outer  space  is 
taking  place  on  a  known  surface  described  by  the  Gaussian  curvature.  Second,  the  initial  motion  conditions 
in  terms  of  object-sensor,  velocity,  accelerations  are  also  supposed  to  be  known.  Third,  the  sensor  array 
may  be  a  plane  or  any  another  smooth  manifold.  Under  these  assumptions,  it  turns  out  to  be  possible  to 
design  continuous  wavelets  in  the  captured  signals  that  estimate  the  actual  motion  parameters  taking  place 
in  the  outer  space  R3  x  R. 

The  PI  has  started  to  re-examine  the  motion  transformations  in  the  context  of  irregular  sampling.  It  is 
clear  that  patterns  in  motion  undergo  a  sampling  which  is  function  of  the  relative  position  of  the  object  with 
respect  to  the  sensor.  This  sampling  varies  from  image  to  image  with  a  smooth  geometric  transformation 
can  be  considered  as  a  warping  operation  on  a  band- limited  function.  This  way  of  approaching  the  problem 
relates  to  Clark’s  theorem  on  frame  reconstruction  formula.  The  PI  has  already  shown  that  rotational  and 
deformational  motions  have  there  own  sampling  properties  and  theorems.  It  seems  that  this  topic  still 
requires  some  more  investigations  since  it  has  never  been  explored  thoroughly  beyond  the  affine  group.  Our 
motion  models  deal  with  extensions  of  the  Galilei  group  for  electronic  sensor  arrays  and  the  Lorentz  group 
for  biological  sensors.  This  topic  interests  the  coding  applications,  and  more  specifically,  the  optimal  coding 
scheme  for  image  sequence  with  areas  of  interest.  Here,  we  assume  that  the  viewer  is  focusing  on  one  single 
moving  target  for  which  he  requires  the  maximum  amount  of  information/resolution  and  that  the  viewer  is 
not  interested  by  anything  else  in  the  content  of  that  scene.  Optimal  coding  scheme  involves  the  following 
sequence:  selectively  unwarping  motion,  decorrelation,  quantization,  entropy  coding.  After  transmission, 
the  inverse  operations  are  taking  place  in  the  decoder. 

The  uncertainty  principle  is  usually  understood  as  a  Heisenberg  inequality  involving  the  wavelet  func¬ 
tions.  There  are  basically  as  many  inequalities  as  there  are  non-commuting  relations  in  the  Lie  algebra. 
This  topic  has  never  been  explored  in  depth  for  a  kinematical  framework.  Moreover,  alternative  approaches 
abound  in  this  still  growing  literature.  In  fact,  besides  the  non-commuting  operator  theory,  this  area  of 
research  can  be  examined  along  two  main  avenues:  using  a  non-complex  point  of  view  already  covered  by 
numerous  theorems  or  exploiting  the  powerful  machinery  of  complex  analysis.  Let  us  mention  that  the 
study  of  important  uncertainties  between  scale,  orientation,  position  and  velocity  have  never  been  investi¬ 
gated  thoroughly  despite  the  fact  that  they  can  be  observed  in  image  sequences.  The  derivation  of  related 
tight  bounds/inequalities  for  motion  estimation  seems  to  be  an  important  topic  to  be  studied. 

The  PI  has  re-examined  the  continuous  wavelet  analysis  in  the  framework  of  the  mathematical  radar  and 
sonar  theory.  In  fact,  the  generalized  ambiguity  function  originally  defined  on  the  Heisenberg  group  is  more 
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than  a  cross-correlation  function  but,  in  fact,  a  continuous  wavelet  transform  where  the  unitary  irreducible 
group  representations  are  square-integrable.  Proceeding  in  that  vein,  we  reach  statistical  detection  theory, 
Weiner  theory,  singularity  expansions  for  target  identification,  Kalman  filtering  and  hidden  Markov  models: 
we  have  already  mentioned  that  the  continuous  wavelet  transforms  smoothly  match  to  those  topics.  If  we 
rephrase  this  approach,  this  project  is  deriving  a  theory  for  passive  radars  which  is  similar  to  that  one 
developed  for  active  microwave  radars.  In  our  case,  the  moving  targets  are  analyzed  from  a  signal  passively 
picked  up  from  natural  radiations.  Moreover,  the  transmission  channel  consists  in  a  projection  on  the  sensor 
array  within  the  cone  of  visibility.  Proceeding  one  step  further,  we  reach  the  theory  of  array  processing  and 
beamforming  which  has  not  been  yet  studied  with  the  point  of  view  of  group  theory. 

A  pending  issue  of  great  interest  that  should  require  more  attention  in  future  research  is  the  area  of 
motion-selective  reconstructions  based  on  the  inverse  wavelet  transform.  To  solve  this  problem,  the  proper¬ 
ties  of  convergence  of  the  inverse  formula  are  of  importance  and  are  closely  related  to  the  image  reconstruction 
technique  using  the  localized  phase  in  the  Fourier  domain.  In  fact,  the  importance  of  the  phase  has  never 
been  investigated  so  far  to  describe  the  motion  content  and  never  related  to  properties  of  weak  or  strong  con¬ 
vergence  of  the  inverse  wavelet  transform  (i.e.  the  corresponding  Calderon  reproducing  formula).  We  may 
also  suppose  that  these  properties  depend  on  the  motion  under  investigation  and  the  characters  of  the  group 
representations.  However,  it  is  well  known  that  under  a  variety  of  conditions,  the  Fourier  phase  is  sufficient 
for  image  representation.  The  earliest  context  in  which  the  Fourier  phase  has  been  recognized  was  for  the 
Fourier  synthesis  of  crystallographic  structures.  Moreover,  computational  results  and  theoretical  analysis 
indicate  that  constrained  and  unconstrained  image  reconstructions  from  local  phase  are  more  efficient  than 
the  reconstruction  from  global  phase  i.e.  computer  operations  are  reduced  and  rates  of  convergence  are 
improved.  Moreover,  the  local  phase  reconstruction  is  also  implementable  with  fast  algorithms  using  highly 
parallel  architectures.  This  topic  remains  as  a  future  path  of  research. 

Another  effort  has  been  devoted  to  characterizing  new  questions  that  may  be  of  importance  for  motion 
analysis: 

1.  develop  the  statistics  of  higher  order,  n-spectra  and  coherence  function  to  study  motion  evolution. 

2.  detect  the  disruptions  of  stationarity  when  a  motion  in  progress  changes  of  kinematic,  is  occluded, 
ends  or  starts. 

3.  introduce  group  representation  theory  in  array  signal  processing. 

4.  investigate  how  information  theory  and  sampling  theory  specifically  contribute  to  motion  transforma¬ 
tions. 

5.  search  alternative  means  of  deriving  ODE’s  and  PDE’s  as  equation  of  motion.  Although  this  topic  can 
be  addressed  with  the  so-called  Laplace-Baltrami  operators,  it  seems  to  remain  a  fairly  open  issue  on 
which  the  PI  will  interact  with  Prof.  Leon  Cohen  at  CUNY. 

6.  relate  item  5  to  build  a  bridge  to  PDE’s  that  encompass  the  research  work  done  for  optical  flow,  and 
for  diffusion  processes. 

7.  examine  the  huge  potentiality  to  create  new  analysis  tools.  Let  us  mention  recent  novelties  like 
‘curvelets’,  ‘ridgelets’  that  should  come  out  in  our  area  as  ‘trajelets’. 

4  Accomplishments/New  Findings 

The  PI  has  showed  the  necessity  to  relate  group  representations  to  one  principle  of  optimality  in  order 
to  thoroughly  formulate  a  selective  tracking  problem.  In  this  vein,  the  PI  exploits  variational  principles 
of  optimality  to  derive  equations  of  wavelet  motion  in  terms  of  Partial  Differential  Equations,  to  derive 
equations  of  tracking  evolution  in  terms  of  Ordinary  Differential  Equations.  The  adjunction  of  a  principle 
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of  optimality  enables  to  determine  the  selective  constants  of  motion  to  be  tracked  and  the  symmetries  to  be 
imposed  to  the  wavelets  in  order  to  track. 

Lie  group  theory  leads  to  harmonic  analysis.  In  our  case,  we  develop  the  harmonic  analysis  related  to 
the  motion  transformations.  The  PI  started  recently  to  investigate  the  related  special  functions  and  their 
relations  with  the  Green  functions  of  the  wavelet  motion  PDE’s.  This  approach  generates  new  integral 
equations  with  Green  functions  as  kernel.  The  study  of  the  properties  of  these  kernel  is  highly  relevant.  The 
PI  has  rediscovered  the  motion-compensated  filters  developed  during  the  period  11-1-93  and  2-1-96  from 
this  recent  research. 

Prom  spatio-temporal  continuous  wavelets,  the  PI  has  determined  the  existence  of  tight  frames  and  or¬ 
thonormal  bases  for  translational  motion.  The  rotational  case  is  also  under  study.  From  these  bases,  the  PI 
will  derive  the  minimal  expansions  of  the  moving  patterns  along  a  trajectory. 

The  premise  relations  between  stochastic  calculus,  stochastic  processes,  Lie  algebras,  Lie  groups,  and 
group  representations  have  been  established  that  introduce  probability  density  on  Lie  generators  or  on  group 
representation.  Brownian  motions  or  diffusion  processes  can  also  be  driven  by  group  representations  and 
drift  along  a  trajectory.  Following  this  approach,  the  Ito  integral  can  generalized  in  this  case.  This  topic  will 
be  subject  to  much  more  attention  since  it  will  allow  to  treat  on  the  same  footing  deterministic,  jittering 
and  diffusing  trajectories. 

At  the  present  stage,  the  PI  has  developed  a  theoretical  framework  that  spans  from  harmonic  analysis 
of  motion  transformation  to  the  differential  geometry  (the  Cartan  geometry)  on  which  motion  occur.  This 
unified  framework  now  reaches  the  present  research  edges  of  complex  analysis,  information  theory,  array 
processing,  statistical  signal  processing,  stochastic  process.  The  research  framework  encompasses  irregular 
sampling,  radar  theory,  signal  decomposition,  inverse  problem  of  image  reconstruction.  As  a  result  of  the 
group-theoretic  foundation  and  the  objective  of  developing  a  smooth  theory,  this  theoretical  framework 
may  now  be  examined  from  various  scientific  points  of  view  without  any  mismatches,  gaps  or  discontinu¬ 
ities.  These  points  of  views  are  as  follows:  optimal  control,  filtering  theory,  differential  geometry,  functional 
analysis,  classical  and  stochastic  mechanics,  relativity,  coding  and  transmission.  As  a  matter  of  fact,  this 
approach  addresses  problems  rooted  in  engineering,  in  mathematical  physics,  in  mechanics,  and  in  pure  and 
applied  mathematics. 

The  PI  has  also  established  smooth  matching  links  between  continuous  wavelet  transform  and  the  group 
representation  theory,  on  one  hand,  and  the  stochastic  and  deterministic  Kalman  theory  on  the  other  hand. 
This  construction  provides  a  broader  interrelated  view  on  Kalman  filtering  which  can  be  stated  in  terms  of 
operator  theory,  stochastic  processes,  estimation  and  filtering  theory.  Depending  on  the  applications,  this 
framework  deals  with  the  motion  tracking,  diffusion  process,  prediction,  interpolation  and  reconstruction. 

Computer  routines  have  been  developed  on  Matlab5  for  the  estimation  and  tracking  of  patterns  moving 
with  jittered  transformations  embedded  in  severe  noisy  pictures  (results  published  on  Journal  and  Conference 
papers).  Comparisons  have  been  presented  with  competing  techniques  based  on  subband  decomposition  and 
block  matching  that  can  not  resolve  the  occlusion  problem.  The  image  sequences  which  have  been  treated 
so  far  contain  translational  and  rotational  velocities,  and  accelerations.  Deformational  motion  has  also  been 
examined  to  measure  the  actual  component  of  velocity  orthogonal  to  the  sensor  plane  i.e.  the  time  before 
collision. 

5  Interactions/Transitions 

The  collaboration  with  Professor  Guido  Weiss  (Mathematics  Department  at  Washington  University  in  Saint 
Louis)  has  progressed  along  the  following  fundamental  topics  successively  related  to  the  Galilean  group 
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and  its  generalizations:  conditions  of  existence  of  continuous  wavelets  and  convergence  of  the  inverse  trans¬ 
form.  Several  important  related  topics  are  going  to  be  covered  as  frames,  orthonormal  bases  i.e.  discrete 
spatio-temporal  wavelets,  spatio-temporal  convolutions,  generalized  Fourier  transforms.  It  has  then  been 
continued  on  the  convergence  problem  of  the  Calderon  reproducing  formula.  This  formula  stands  as  the 
wavelet  condition  of  admissibility.  This  condition  defines  the  continuous  wavelet  transform  as  an  isometry 
operator  associated  with  a  set  of  well  known  properties.  The  convergence  of  the  reproducing  formula  can 
be  strong  or  weak  according  to  the  group  under  investigation  and  so  is  the  topology.  This  topic  is  supposed 
to  provide  a  theoretical  support  on  the  issue  of  the  motion  selective  reconstructions. 

The  collaboration  with  Professor  Victor  Wickerhauser  (Mathematics  Department  at  Washington  Uni¬ 
versity  in  Saint  Louis)  has  covered  several  topics:  reproducing  kernels,  Euler-Lagrange  equations,  square- 
integrability  over  the  different  motion  groups  of  interest. 

The  collaboration  with  Professor  Brian  Blank  (Mathematics  Department  at  Washington  University  in 
Saint  Louis)  has  focused  on  the  construction  of  special  functions  related  to  the  group  representations, 
especially  for  rotational  motion  (trigonometric  and  hyperbolic  rotations),  and  for  deformational  motion. 
Although  being  different,  these  special  functions  behave  like  Bessel  functions. 

Lively  interactions  with  Professor  John  Benedetto  (Mathematics  Department  at  University  of  Maryland) 
have  been  initiated  and  will  turn  into  a  collaboration  devoted  on  investigating  the  potential  issues  of  inter¬ 
est  that  may  not  have  been  addressed  so  far  in  this  research  program:  topics  like  the  irregular  sampling 
related  to  motion  transformations  and  warpings,  like  the  construction  of  new  analytic  expressions  of  dis¬ 
crete  wavelet  transforms  associated  with  the  group  of  motion  under  study,  like  new  group  representations 
in  complex  functional  spaces. 

Lively  interactions  have  been  realized  with  Professor  Bijoy  Ghosh.  (Systems  Science  &;  Mathemat¬ 
ics  Department  at  Washington  University  in  Saint  Louis)  Professor  B.  Ghosh  is  supporting  all  my  heavy 
computational  efforts.  Discussions  with  Professor  Bijoy  Ghosh  have  dealt  with  the  applications  of  motion 
estimation  in  robotics.  Professor  Ghosh  has  been  very  interested  by  the  potentiality  of  having  at  hand 
new  numerical  tools  that  can  estimate  motion  parameters  like  angular  and  deformation  velocity.  The  fact 
of  being  able  to  estimate  in  captured  signal  the  actual  motion  parameters  taking  place  on  known  surfaces 
in  the  outer  space  seems  to  be  of  tremendous  importance  for  robotic  applications.  The  motion  detection, 
estimation  and  tracking  is  one  of  the  most  important  property  of  robotic  vision  must  have.  The  efficiency 
of  these  estimation  techniques  will  tremendously  influence  the  autonomous  capacity  of  a  robot. 

Lively  interactions  with  Professor  Lawrence  Conlon  (Mathematics  Department  at  Washington  University  . 
in  Saint  Louis)  have  been  pursued  during  the  past  academic  year  about  differential  geometry.  The  purpose 
was  to  understand  how  geometers  tackle  the  problem  of  generating  and  transporting  bases  on  smooth 
manifolds  and  geodesics.  For  instance,  how  to  properly  define  an  orthonormal  basis  on  some  Riemannian 
geometry  and  how  to  process  it  in  a  parallel  transportation  on  a  geodesic.  This  subject  is  a  generalized 
version  of  how  to  design  discrete  wavelets  and  how  to  displace  them  along  the  assumed  trajectory  of  motion. 
Both  ‘top  down’  (by  the  differential  geometry)  and  ‘bottom  up’  (by  the  wavelet  theory)  approaches  are 
expected  to  match. 

Lively  interactions  with  Professor  Renato  Feres  (Mathematics  Department  at  Washington  University  in 
Saint  Louis)  have  help  to  develop  the  means  to  introduce  noise  generators  in  the  Lie  theory.  Noise  and  ran¬ 
dom  processes  can  be  introduced  in  the  algebra,  the  group  and  in  the  group  representations  in  order  to  yield 
the  appropriate  stochastic  calculus  that  combines  diffusion  process  and  motion  transformation.  Restated  in 
ather  words,  problem  at  hand  consisted  on  how  to  model  a  diffusion  process  on  a  motion  trajectory.  This 
research  effort  leads  towards  stochastic  PDE’s  and  integral  equations  of  the  Ito  family.  This  topics  of  com¬ 
bining  stochastic  processes  and  group  theory  has  also  been  a  topic  of  interest  in  the  Department  of  Electrical 
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Engineering  at  Washington  University  in  Saint  Louis  where  Professor  J.  O’Sullivan  is  our  interlocutor. 

Lively  interactions  have  been  initiated  with  Professor  Welland  (Mathematics  Department  of  the  Univer¬ 
sity  of  Missouri  at  Saint  Louis)  on  alternative  ways  of  tackling  the  problem  of  motion  analysis.  We  adopted 
the  perspective  of  biological  and  human  perception  properties.  Biological  sensors  behave  differently  from 
the  electronic  sensor  arrays.  Since  biological  sensors  rescale  time  and  space,  we  are  conducted  to  considering 
other  Lie  groups  based  on  the  Lorentz  transformations.  In  fact,  this  biological  approach  of  motion  analysis 
based  on  the  Human  Visual  System  has  been  a  topic  studied  with  group  representations  before  the  author 
started  this  research  program.  Among  these  previous  works,  let  us  mention  authors  like  Caeli  (1978),  Zeevi 
(1992),  Duval-Destin  and  Murenzi  (1992). 

Lively  interactions  with  Professor  Mohan  Kumar  (Mathematics  Department  at  Washington  University 
in  Saint  Louis)  have  covered  the  topic  of  how  abstract  algebra  can  help  to  develop  fast  algorithms.  It  is 
well  known  that  the  Fast  Fourier  Transform  developed  by  Cooley-Tucker  relies  on  algebraic  structures  like 
the  Chinese-remainders  theorem,  the  Poisson  formula  and  some  properties  of  the  group  characters.  The 
polynomial  ring  theory  is  central  to  many  applications  at  hand  for  both  multidimensional  convolution  and 
multidimensional  Fourier  transform.  This  approach  is  related  to  the  H.  J.  Nussbaumer’s  work  where  poly¬ 
nomial  transforms  use  the  Chinese  remainders  to  decompose  the  global  computation  into  subcomputations. 
This  discussion  is  primordial  to  be  aware  of  how  fast  algorithms  may  be  implemented,  and  therefore,  to  be 
aware  of  how  to  set  up  appropriate  structures  and  ideas  that  lead  to  algorithms  of  estimation,  tracking  and 
reconstruction  allowing  fast  parallel  computations. 

During  the  period  9-1-97  to  5-1-99,  the  PI  ideas  have  supported  the  writing  of  a  Ph.D.  dissertation  for 
a  recipient  student  in  the  Mathematics  Department  of  Washington  University  in  Saint  Louis.  It  was  an 
actual  opportunity  for  this  Ph.D.  student,  Jon  Corbett,  to  learn  the  theory  of  group  representations  using 
Mackey’s  theory  and  Kirrilov’s  technique  of  orbits,  to  learn  the  mathematical  physics  concerning  the  Galilei 
and  Poincare  groups  as  well  as  the  properties  of  the  continuous  wavelets  over  several  groups  of  motion 
transformation.  The  Ph.D.  dissertation  of  Dr.  Jon  Corbett  is  provided  in  annex  of  this  final  report. 

During  the  period  9-1-96  to  9-1-97,  the  PI  has  developed  an  application  of  the  continuous  wavelet  trans¬ 
forms  that  performs  automatic  target  recognition  co-advising  a  graduate  student  F.  Mujica  at  the  Georgia 
Institute  of  Technology.  The  particular  problem  that  has  been  addressed  in  this  research  is  the  tracking 
a  warhead  in  an  incoming  ballistic  missile.  The  scenario  assumes  that  the  ballistic  missile  has  fragmented 
and  that  the  wearhead  is  among  the  missile  fragments.  The  task  at  hand  is  to  develop  a  robust  algorithm 
that  will  allow  the  interceptor  to  track  and  neutralize  the  threat.  An  optical  sensor  housed  in  the  inter¬ 
ceptor’s  nose  cone  provides  the  images  that  are  processed  and  used  to  direct  the  course  of  interception. 
The  main  practical  focus  in  this  application  is  the  selective  tracking  of  small  moving  objects  embedded  in 
digital  image  sequences  and  the  construction  of  their  trajectories.  The  digital  signals  that  originate  from 
the  sensors  are  sampled  at  a  rate  of  100  images  per  second.  Realistic  assumptions  have  been  considered 
for  motion  and  noisy  data.  The  motion  to  be  tracked  is  not  only  linear  and  uniformly  accelerated  but  also 
jittered  by  a  random  motion  on  the  sensors  (Gaussian  noise  of  standard  deviation  of  one  pixel  away  from 
the  trajectory).  The  image  quality  can  also  degrade  quite  significantly  to  display  poor  signal-to-noise  ratio 
less  than  lOdB  signal-to-noise  ratio.  The  algorithm  to  be  developed  should  work  as  a  framework  associating 
continuous  wavelet  transforms  and  signal  representations  with  other  efficient  techniques  in  order  to  perform 
motion  and  velocity  detection,  measurements  or  estimations  and  selective  tracking.  The  Ph.D.  dissertation 
of  Dr.  Fernando  Mujica  can  be  obtained  at  library  of  the  Georgia  Institute  of  Technology  under  the  title: 
“Spatio-temporal  continuous  wavelet  transform  for  motion  estimation”,  1999. 

During  the  period  9-1-97  to  5-1-99,  the  PI  ideas  on  how  to  apply  the  Galilean  wavelets  to  perform 
motion-based  segmentations  of  digital  scenes  have  been  implemented  by  Mingqi  Kong  a  Ph.D.  student 
in  the  Systems-Sciences-Mathematics  Department  of  Washington  University  in  Saint  Louis.  As  expected 
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the  continuous  wavelet  transform  is  efficient  to  sketch  all  the  spatio-temporal  edges  or  discontinuities  i.e. 
the  tubes  of  motion.  In  this  approach,  we  get  on  the  same  footing  the  edges  and  the  estimation  of  the 
motion  parameters  (this  is  not  the  case  in  classical  techniques  where  edges  are  required  to  estimate  motion 
and  vice  versa).  This  approach  may  then  successfully  simplify  the  usual  algorithms  based  on  operation 
research  techniques  specially  for  target  pursuit.  The  Ph.D.  dissertation  of  Mingqi  Kong  can  be  found  on 
the  dissertation  abstract  on  line  and  ordered  at  the  library  of  Washington  University  in  Saint  Louis  under 
the  title;  “Motion  Estimation  and  Motion-Based  Segmentation  in  Digital  Image  Sequences”,  1999. 

Paper  presentations  at  Conferences  and  Seminars: 

Presentation  of  seminars  after  invitation  in  the  department  of  electrical  engineering  at  the  New  Jersey  In¬ 
stitute  of  Technology  (New  Jersey,  1996),  Polytechnic  Institute  of  Brooklyn  (New  York,  1997),  Rochester 
University  (New  York  state,  1996),  Rensselear  Polytechnic  Institute  (New  York  state,  1998),  University  of 
Texas,  San  Antonio  (Texas,  1997). 

Presentation  of  seminars  after  invitation  in  the  department  of  sciences,  systems  and  mathematics  at  the 
Washington  University  in  Saint  Louis  (Missouri,  1998). 

Presentation  of  seminars  after  invitation  in  the  department  of  mathematics  at  the  Texas  A&M  University, 
College  Station  (Texas,  1996),  University  of  Maryland  at  College  Park  (Maryland,  1996),  Rensselear  Poly¬ 
technic  Institute  (New  York  state,  1998),  University  of  Missouri  in  Saint  Louis  (Missouri,  2000). 

The  research  has  also  been  presented  at  the  ’’Institut  de  Recherche  en  Informatique  et  Systemes  Aleatoires 
(IRIS A,  Campus  de  Beaulieu,  Rennes,  France):  “Estimation  et  pistage  du  mouvement  par  ondelettes  con¬ 
tinues  spatio-temporelles”,  1995. 

1.  ICIP-98,  Chicago,  October  5,  1998  (two  papers). 

2.  Time-Frequency  and  Time-Scale  Analysis:  Pittsburgh,  October  6,  1998  (1  paper). 

3.  IT  Workshop  on  Detection,  Estimation,  Classification  and  Imaging:  Santa  Fe, 

February  26,  1999  (1  paper). 

4.  ICASSP-99,  Phoenix,  March  16,  1999  (1  paper). 

5.  two  presentations  in  the  Mathematics  department  on  Lie  group  representation  theory:  “Properties  of  Co¬ 
herent  States  and  Continuous  Wavelets”,  1997,  and  “Group  Representation  Theory  for  Nilpotent,  Solvable 
and  Exponential-Solvable  Groups”,  1998. 

6.  one  presentation  in  the  department  of  Systems  Science  and  Mathematics  at  Washington  University  in 
Saint  Louis  about  the  PI  on-going  research. 

7.  ICASSP-2000,  Istanbul,  Turkey,  June  5-9  2000  (1  paper). 

8.  NATO-ASI-Conference:  20th  Century  Harmonic  Analysis,  A  Celebration,  II  Ciocco,  Italy,  July  2-15  2000 
(1  poster). 

9.  IEEE  Workshop  on  Statistical  Signal  and  Array  Processing  Pocono  Manor,  Pennsylvania,  August  14-16 
2000  (1  paper). 

10.  IEEE  International  Symposium  on  Information  Theory,  Sorrento,  Italy,  25-30  June  2000  (1  paper). 

11.  invited  presentation  in  the  department  of  Electrical  Engineering  at  Washington  University  in  Saint  Louis. 
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12.  presentation  at  the  Wavelet  and  Harmonic  Analysis  seminar  of  the  Mathematics  department  of  the 
University  of  Maryland  :  “Group-Theoretic  Approach  for  the  Harmonic  Analysis  of  Spatio-Temporal  Trans¬ 
formations”,  December  2000. 

13.  IEEE  International  Symposium  on  Information  Theory,  Washington  DC,  24-29  June  2001  (1  paper). 


6  Further  Paths  for  Research 

The  further  research  will  focus  on  developing  a  unified  theoretical  framework  that  deals  with  harmonic 
analysis  associated  with  motion  analysis.  We  shall  derive  new  mathematical  tools  to  process  spatio-temporal 
digital  signals,  new  algorithms  for  optimal  motion  estimation  and  tracking,  and  new  fast  parallelizable  hard¬ 
ware  implementations  similar  to  the  Cooley-Tukey  algebraic  structures  or  the  quantum  Fourier  transform. 
The  first  principle  is  to  model  local  motion  transformations  by  Lie  algebras.  Lie  algebras  characterize  both 
smooth  manifolds  and  motion  in  an  way  which  is  easy  to  handle.  The  construction  starts  with  Lie  algebras 
and  proceeds  to  Lie  groups,  and  then  to  group  representations  on  function  spaces,  e.g.,  Hilbert  spaces  are 
often  appropriate  for  signal  analysis.  This  construction  establishes  a  connection  between  motion  on  differ¬ 
entiable  manifolds  and  operators  acting  on  functions.  Some  properties  of  these  operators  allow  us  to  derive 
motion-based  correlators,  as  well  as  continuous  and  discrete  wavelet  transforms  required  for  effective  motion 
analysis.  This  is  the  foundation  of  our  entire  theoretical  framework. 

Our  mathematical  approach  will  incorporate  the  physical  nature  of  motion  in  digital  signal  analysis, 
and  in  human  visual  perception.  Further,  a  variational  principle  of  optimality ,  based  on  Lagrangians  or 
Hamiltonians,  is  required  to  link  all  the  local  transformations  into  trajectories.  This  provides  tracking 
abilities  and  trajectory  constructions  with  fast  algorithms  from  dynamic  programming.  This  theoretical 
framework  generalizes  all  the  current  techniques  of  motion  and  diffusion  analysis  presented  in  the  literature. 
Bayesian  and  non-Bayesian  statistical  inferences  have  to  be  adjoined  to  our  Lie-theoretic  construction  for 
optimal  estimation  and  tracking  purposes. 

Consider  motion  taking  place  in  I3  x  E  (3-dimensional  space  and  time).  In  this  case,  for  the  purpose 
of  analysis,  motion  is  projected  onto  sensor  arrays  and  captured  as  apparent  motion  in  R2  x  E  digital  signals 
or  in  the  human  visual  system.  The  project  investigates  motion  and  sensors  deployed  on  smooth  manifolds, 
and  the  projective  transformations  that  relate  them.  The  set  of  motion  transformations  under  consider¬ 
ation  contains  translation,  rotations,  and  dilation  along  with  their  temporal  derivatives,  i.e.,  velocity  and 
acceleration.  Stochastic  processes  can  be  introduced  at  different  levels  of  the  theory  to  consider  stochastic 
geometry,  random  array  sensors,  jittered  motion,  and  diffusion  processes. 

The  construction  of  this  unified  framework  requires  the  following  research. 

1.  Generate  new  Lie  algebras,  Lie  groups,  group  representations,  and  discrete  and  continuous  wavelet 
transforms,  and  introduce  stochastic  processes  at  the  appropriate  stages.  These  new  algebras  are  necessary 
to  treat  motion  on  smooth  manifolds  associated  with  tracking  and  estimation  for  applications  dealing  with 
non-planar  sensor  arrays. 

2.  Investigate  the  properties  of  the  group  characters  and  design  fast  parallelizable  hardware  imple¬ 
mentation  of  the  special  functions  associated  with  these  characters. 

3.  Derive  the  uncertainty  principle  required  to  obtain  the  optimal  strategy  for  estimation  and  tracking. 

4.  Exploit  operator  theory,  stochastic  calculus,  and  optimal  control  in  order  to  generate  new  sets  of 
ODEs  and  PDEs  that  govern  selective  tracking  and  diffusion  processes.  Formulate  new  orthogonal  bases 
and  convolutional  motion-compensated  filters.  These  new  models  will  determine  equations  of  the  selective 
tracking  along  with  their  corresponding  tools  of  analysis. 

5.  Investigate  the  irregular  sampling  problem  related  to  random  array  sensors,  the  human  visual 
system,  and  motion-selective  reconstruction. 


9 


6.  Develop  the  theoretical  framework  with  models,  a  DSP  toolbox,  and  fast  parallelizable  algorithms 
to  treat  motion  analysis  in  spatio-temporal  digital  signals  and  in  the  human  visual  system. 

To  outline  the  main  orientations  of  this  theoretical  research,  we  can  see  that  the  group-theoretic 
approach  remarkably  provides  all  the  appropriate  analytical  formulations  related  to  motion  analysis  and 
diffusion  process,  the  whole  support  to  perform  optimal  tracking  (uncertainties,  correlators,  estimators, 
PDE’s  and  ODE’s),  and  the  structure  of  the  fast  algorithms  for  motion  estimation  and  reconstruction. 

7  Conclusions 

8  Publications 

8.1  Publications  in  Conferences 

1.  [17]  J.-P.  Leduc,  J.-M.  Odobez,  C.  Labit,  “ Adaptive  Motion-Compensated  Wavelet  Filtering  for  Image 
Sequence  Processing?' ,  ICASSP-95,  Detroit,  USA,  8-12  May  1995,  Vol.  4,  pp.  2367-2370. 

2.  J.-P.  Leduc  and  C.  Labit,  “ Spatio-Temporal  Wavelet  Transforms  for  Image  Sequence  Analysis",  VIII  Eu¬ 
ropean  Signal  Processing  Conference,  EUSIPCO-96,  Trieste,  Italy,  10-13  September  1996,  4  pp. 

3.  J.-P.  Leduc  and  C.  Labit,  “ Spatio-Temporal  Wavelet  Transforms  for  Motion  Analysis ",  SPIE’s  Wavelet 
Applications  in  Signal  Processing  IV,  Boulder,  USA,  4-9  August  1996. 

4.  J.-P.  Leduc  ,  “ Spatio-Temporal  Wavelet  Transforms  for  motion  tracking',  ICASSP-97,  Munich,  Germany, 
20-24  April  1997,  Vol.  4,  pp.  3013-3017. 

5.  F.  Mujica,  J.-P.  Leduc,  R.  Murenzi,  M.  J.  S.  Smith,  “Spatio-Temporal  Continuous  Wavelets  applied  to 
missile  warhead  detection  and  tracking',  SPIE’s  Visual  Communications  and  Image  Processing  1997,  San 
Jose,  California,  8-14  February  1997,  12  pp. 

6.  F.  Mujica,  J.-P.  Leduc,  R.  Murenzi,  M.  J.  S.  Smith,  “ Missile  Tracking  Algorithm  Using  Target-Adapted 
Spatio-Temporal  Wavelets",  SPIE  AeroSense’97,  Orlando,  April  1997,  12  pp. 

7.  F.  Mujica,  J.-P.  Leduc,  R.  Murenzi,  M.  J.  S.  Smith,  “Target  Tracking  in  the  Presence  of  Flare  Decoys  Using 
the  Spatio-Temporal  Wavelet  Transform",  Proceedings  of  the  1998  ARL  Sensors  and  Electronic  Devices 
Symposium,  College  Park,  February  1998. 
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9  Annexes 

Additional  pieces  of  information  to  support  the  understanding  and  the  transfer  of  research  related  to  this 
project  axe  provided  as  follows  in  6  annexes: 

1.  the  group-theoretic  framework  is  outlined  in  annexe  1. 
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2.  the  Ph.D.  thesis  dissertation  in  mathematics  of  Dr.  Jonathan  Corbett,  a  more  precise  and  intimately 
related  material  can  be  find  in  the  following  thesis  of  C.  Kalisa  ”Etats  Coherents  Affines:  Canoniques, 
Galileens  et  Relativistes’\Ph.D.  thesis,  Universite  Catholique  de  Louvain,  1993.  These  theses  provide 
similar  computation  of  Lie  group  representations  associated  to  computations  of  kinematical  groups 
like  the  Galilei  group. 

3.  a  paper  under  review  dealing  with  accelerated  wavelets. 

4.  a  paper  under  review  dealing  with  rotational  wavelets. 

5.  demonstration  routines  on  Matlab5.  Most  of  the  explanations  on  how  to  deal  with  those  algorithms  is 
provided  in  two  published  papers:  (1)  sf  F.  Mujica,  J.-P.  Leduc,  R.  Murenzi,  and  M.  Smith  “A  New 
Parameter  Estimation  Algorithm  Based  on  the  Continuous  Wavelet  Transform”,  IEEE  Transactions 
on  Image  Processing,  Vol.  9,  No.  5,  May  2000,  pp.  873-888,  and  in  paper  (2)  F.  Mujica,  R.  Murenzi, 
M.  Smith  and  J.-P.  Leduc  “A  New  Object  fracking  Algorithm  Based  on  The  Continuous  Wavelet 
Transform",  SPIE  Journal  of  Electronic  Imaging,  pp.  746-754,  Vol.  7,  October  1998 

6.  additional  helpful  references  from  the  literature 
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